In datacenters, a multitude of network devices may act as intermediary proxies to facilitate communications between client devices and backend servers. Such network devices may exist behind a limited number of public Internet Protocol (IP) addresses facing toward the client devices. Since the datacenter may only have limited number of public IP addresses, each of these network devices may not be assigned to an individual public IP address. With a limited number of public IP addresses, incoming and outgoing communications between the client devices and backend servers in the datacenter may need to be load balanced among the multitude of network devices. Conventional approaches to load balancing communications in such network environments may result in high latency and consumption of bandwidth and computing resources, thereby degrading the overall performance of the datacenter.